The invention relates generally to detection of messages exchanged between processes in a computer system and in particular to creating thread-level message hooks for detecting messages exchanged with a specific process or thread.
Modern operating systems, such as recent versions of the Microsoft Windows operating system, are “multitasking,” meaning that they generally execute a number of processes in parallel. For example, in Windows XP, it is typical for two or three dozen processes (or more) to be executing at a given time. These processes usually include numerous operating system processes, device drivers, and the like, as well as “application” processes, such as video games, World Wide Web browsers, e-mail or instant messaging clients, and other programs that are not integrated into the operating system. Application processes typically interact with the operating system by exchanging messages with operating system processes. For instance, when a Web browser process creates a new window for displaying a Web page, the browser process usually sends a message to the operating system requesting a new window; the operating system responds with another message when the window is created. User input, such as keystrokes or mouse clicks, is usually received by the operating system, which responds by sending messages to one or more application processes.
A typical message flow for a conventional computer system is shown in FIG. 1A, with circled numbers next to the arrows indicating a typical sequence of message transmissions. An operating system (OS) 102 is represented as a single process, although in practice, the OS may include any number of processes that may exchange messages with each other and/or with application processes. Two application processes 104 are shown, and each process has its own message queue 108 for temporarily storing messages received from OS 102. (Herein, multiple instances of like objects are denoted with reference numbers identifying the object and parenthetical numbers identifying the instance where appropriate.) Each application process 104 retrieves messages sequentially from its queue 108 and takes appropriate action in response to each message. In some instances, the action may include generating a message to the OS, which can also be queued in the application's message queue 108. Message queues 108 enable asynchronous operation of OS 102 and application processes 104. For instance, OS 102 can generate a message for application process 104(1), place it in queue 108(1), and proceed to another activity without waiting for application process 104(1) to receive or respond to the message. Similarly, process 104(1) can generate a message for OS 102, place it in queue 108(1) and proceed without waiting for a response.
FIG. 1A shows that a separate message queue 108 may be provided for each application process 104. This allows processes to operate independently of each other; for example, application process 104(2) can receive and process any messages in its queue even if application process 104(1) stalls or crashes. While this configuration has many advantages, it is generally not possible for one application process to detect messages exchanged between the OS and another application process. For instance, in FIG. 1A, application process 104(2) cannot detect messages in queue 108(1). Thus, application process 104(2) cannot reliably detect when an event (such as a window being created or destroyed) occurs in process 104(1).
Many applications do not require such information, but there are various kinds of applications for which knowledge of events occurring in other processes is desirable or necessary. One example is a “desktop manager” application that provides enhanced user control of the graphical desktop. A desktop manager typically supports functionality such as displaying the desktop across multiple display screens (with the ability to drag windows from one screen to another) or making windows appear transparent or translucent on demand, and may also support various customizations and esthetic effects, e.g., animations or skins. In order to create the appropriate effects, the desktop manager generally needs to know when another application's windows are being created, moved, or destroyed.
To enable one process (e.g., a desktop manager) to detect messages exchanged with other processes, some operating systems provide global system message hooks. A global system message hook intercepts all messages (or selected messages) between the OS and any application program, thereby enabling a hook procedure to act on the message. In some instances, the hook procedure may alter the message or prevent delivery of the message to the recipient process. Message flow in a conventional computer system with a global system message hook is illustrated in FIG. 1B, with circled numbers again indicating an order of message transmissions. When OS 102 generates a message to application process 104(1), it sends the message first to a hook procedure 110. Hook procedure 110, which is typically supplied by an application programmer, performs its programmed function(s) and returns a result to OS 102. The result may include the original message or an altered version of the message to be forwarded to application process 104(1), or an instruction to OS 102 not to deliver the message to application process 104(1). Where the hook procedure does not block message delivery, OS 102 places the message into a systemwide message queue 112 from which it is retrieved by the appropriate application process, in this case process 104(1). In this implementation, when a global system message hook is present, process-specific queues 108 (shown in FIG. 1A) are not used; they are replaced by systemwide message queue 112.
It should be noted that, although the global system message hook of FIG. 1B is able to detect messages related to any application that may be executing, it also has significant disadvantages for system performance. First, messages for all application programs must be processed by the hook procedure before they are delivered, even though the programmer of the hook procedure may only be interested in messages related to certain application programs. Second, if the hook procedure freezes or crashes, other applications generally also freeze or crash since they are unable to receive messages. Third, all messages are serialized in a single systemwide message queue 112, and applications contend with each other for access; a stalled application can block access by other applications. Such considerations have led some operating system vendors to recommend that developers limit their use of global system message hooks.
As an alternative approach to monitoring messages, some operating systems support “thread-level” message hooks. These hooks are installed within a process (or a thread if the OS supports multi-threaded processes) and affect only that process or thread. An example is shown in FIG. 1C, where a thread-level message hook has been installed for application process 104(2) but not for application process 104(1); again, the circled numbers next to the arrows indicate an order of message transmissions that may occur. Messages from OS 102 to process 104(2) are first sent to a thread-level hook procedure 114(2), which returns a result to OS 102. The message is then added to thread-specific message queue 108(2), from which application process 104(2) retrieves the message. Messages from OS 102 to process 104(1) are not sent to thread-level hook procedure 114(2); instead, they are sent directly to the thread-specific queue 108(1) for application process 104(1). A separate thread-level hook procedure (not shown) could be installed for application process 104(2) if desired. While this configuration is more robust than the global system message hook configuration of FIG. 1B, it does not enable application process 104(1) to detect messages that may be directed to or from application process 104(2) because thread-level hook procedure 114 is effectively internal to application process 104(2) and does not communicate with application process 104(1).
Some operating systems, such as Microsoft Windows, support computer-based training by enabling one application to be notified when another application starts or stops. This can be implemented using a “global notification hook,” which generally includes a hook procedure that receives selected messages for all application processes (such as start and stop messages) but is not in the message path between the OS and any application process. An example of message flows in a computer system with a global notification hook is shown in FIG. 1D, with circled numbers indicating an order of message transmissions that may occur; circled numbers followed by letters indicate message transmissions that may occur in parallel. An OS 102 transmits a message to an application process (or thread) 104(1) by placing the message in a process-specific (or thread-specific) message queue 108(1) for process 104(1); process 104(1) retrieves the message from queue 108(1). In parallel with placing the message in queue 108(1), OS 102 transmits the message to a global notification hook procedure 118. Similarly, OS 102 can transmit a message to a different application process 104(2) by placing the message in a process-specific queue 108(2) for process 104(2) and, in parallel, transmitting the message to global notification hook procedure 110. Thus, global notification hook procedure 110 may receive messages for all processes without being in the message path between the OS and any application process.
Since the global notification hook procedure 118 is not in the message path, some features associated with message hooks are absent. For instance, a global notification hook cannot modify a message or block transmission of the message. In the context of a desktop management program, these features are sometimes useful, and a global notification hook does not provide them. Thus, existing desktop management programs rely on global system message hooks despite their drawbacks.
It would, therefore, be desirable to provide a way for a first process in a multitasking system to interact with a second process without the disadvantages of a global system message hook.